382 research outputs found
Communication-Efficient Federated Bilevel Optimization with Local and Global Lower Level Problems
Bilevel Optimization has witnessed notable progress recently with new
emerging efficient algorithms, yet it is underexplored in the Federated
Learning setting. It is unclear how the challenges of Federated Learning affect
the convergence of bilevel algorithms. In this work, we study Federated Bilevel
Optimization problems. We first propose the FedBiO algorithm that solves the
hyper-gradient estimation problem efficiently, then we propose FedBiOAcc to
accelerate FedBiO. FedBiO has communication complexity
with linear speed up, while FedBiOAcc achieves communication complexity
, sample complexity and also the linear
speed up. We also study Federated Bilevel Optimization problems with local
lower level problems, and prove that FedBiO and FedBiOAcc converges at the same
rate with some modification.Comment: arXiv admin note: text overlap with arXiv:2205.0160
Compositional Federated Learning: Applications in Distributionally Robust Averaging and Meta Learning
In the paper, we propose an effective and efficient Compositional Federated
Learning (ComFedL) algorithm for solving a new compositional Federated Learning
(FL) framework, which frequently appears in many machine learning problems with
a hierarchical structure such as distributionally robust federated learning and
model-agnostic meta learning (MAML). Moreover, we study the convergence
analysis of our ComFedL algorithm under some mild conditions, and prove that it
achieves a fast convergence rate of , where denotes
the number of iteration. To the best of our knowledge, our algorithm is the
first work to bridge federated learning with composition stochastic
optimization. In particular, we first transform the distributionally robust FL
(i.e., a minimax optimization problem) into a simple composition optimization
problem by using KL divergence regularization. At the same time, we also first
transform the distribution-agnostic MAML problem (i.e., a minimax optimization
problem) into a simple composition optimization problem. Finally, we apply two
popular machine learning tasks, i.e., distributionally robust FL and MAML to
demonstrate the effectiveness of our algorithm.Comment: 21 pages, 8 figure
How to Train Your Dragon: Tamed Warping Network for Semantic Video Segmentation
Real-time semantic segmentation on high-resolution videos is challenging due
to the strict requirements of speed. Recent approaches have utilized the
inter-frame continuity to reduce redundant computation by warping the feature
maps across adjacent frames, greatly speeding up the inference phase. However,
their accuracy drops significantly owing to the imprecise motion estimation and
error accumulation. In this paper, we propose to introduce a simple and
effective correction stage right after the warping stage to form a framework
named Tamed Warping Network (TWNet), aiming to improve the accuracy and
robustness of warping-based models. The experimental results on the Cityscapes
dataset show that with the correction, the accuracy (mIoU) significantly
increases from 67.3% to 71.6%, and the speed edges down from 65.5 FPS to 61.8
FPS. For non-rigid categories such as "human" and "object", the improvements of
IoU are even higher than 18 percentage points
- …